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EXECUTIVE SUMMARY 

The Youth Transition Demonstration (YTD) is a large-scale demonstration and evaluation 
sponsored by the Social Security Administration (SSA) to improve understanding of how to help 
youth with disabilities reach their full economic potential. In particular, SSA is interested in 
developing and testing promising approaches for helping young people with disabilities become 
more self-sufficient and less reliant on disability benefits. The YTD conceptual framework, which is 
based on best practices in facilitating youth transition, specifies that the six projects participating in 
the evaluation provide employment services (emphasizing paid competitive employment), benefits 
counseling, links to services available in the community, and other assistance to youth with 
disabilities and their families. Additionally, participating youth are eligible for SSA waivers of certain 
benefit program rules, which allow them to retain more of their disability benefits and health 
insurance while they work for pay. Using a rigorous random assignment methodology, the YTD 
evaluation team is assessing whether these services and incentives are effective in helping youth with 
disabilities achieve greater independence and economic self-sufficiency. The earliest of the 
evaluation projects began operations in 2006 and ended in 2009. The latest started in 2008 and will 
end in 2012. 

In this report, we present first-year evaluation findings for the Transition WORKS YTD 
project, which served youth disability beneficiaries in Erie County, New York, including the city of 
Buffalo, from February 2007 to December 2009. While it will take several more years before we fully 
observe the transitions that youth participants make to adult life, early data from the evaluation 
provide rich information on how Transition WORKS operated and the differences it made in key 
outcomes for youth. Specifically, the report includes findings from our process analysis of Transition 
WORKS, including a description of the program model, how the project was implemented and 
services were delivered, and the project’s fidelity to the YTD model. The report also includes impact 
findings, based on data collected 12 months after youth entered the evaluation, on the use of 
services, paid employment, participation in education, income from earnings and benefits, and 
attitudes and expectations. 

In brief, we learned that Transition WORKS was a well-organized, cohesive project that broadly 
conformed to the YTD program model and focused on self-determination, benefits planning, 
employment, education, and case management. Through the process analysis, we found that the 
project enrolled 83 percent of eligible youth and provided most of them with some services in each 
of these components. The median duration of services directly delivered to participating youth was 
lower for the employment component than for several of the other components; however, the 
impact analysis found that youth who had been given the opportunity to participate in Transition 
WORKS were more likely to have used services to promote employment than youth in a randomly 
selected control group. Nevertheless, we found no impacts of the project on youth employment 
during the year following random assignment. Neither did we find impacts on income, expectations 
for the future, and a composite measure of school enrollment or high school completion. 



1 In 2005, under SSA contract #SS00-05-60084, Mathematica Policy Research, a nonpartisan firm that conducts 
policy research and surveys, and its partner organizations, MDRC and TransCen, Inc., were awarded a contract to design 
and conduct the YTD evaluation and provide technical assistance to projects as they developed and implemented their 
interventions. The evaluation is advised by a technical working group consisting of young adults with disabilities, 
providers of services to teenagers and young adults with disabilities, policy researchers, academics, and representatives of 
federal agencies other than SSA. 



xv 



Interim Report on Transition WORKS 



Executive Summary 



The Youth Transition Demonstration Evaluation 

The target population for the YTD evaluation is youth ages 14 through 25 who are either 
receiving SSA disability benefits or are at risk of receiving them in the future. The evaluation is 
based on a rigorous random assignment design. Youth who agree to participate in the evaluation are 
assigned at random to a treatment or control group. Youth in the treatment group are eligible to 
receive YTD services in addition to the SSA waivers, while those in the control group may receive 
only those services available in their communities, independent of the YTD initiative. The evaluation 
seeks to enroll approximately 880 youth in each of the six project sites. 

We gathered information from a variety of sources to inform the findings in this report. We 
obtained information about project operations and the service environment through reviews of 
project documents, site visits, interviews with managers and staff, and focus group discussions with 
participating youth and their parents. We also examined data on enrollment of youth and service 
provision in the project’s management information system. Data for the impact analysis came from a 
12-month follow-up survey and SSA administrative records. The survey focused on outcomes such 
as service use, employment, earnings, education, and attitudes and expectations. SSA administrative 
records provided data on benefits and the use of SSA work incentives and waivers. We also collected 
baseline data prior to random assignment through a survey and SSA administrative records. The 
comprehensive final report on the YTD evaluation, scheduled for 2014, will use data from a survey 
conducted 36 months after random assignment and SSA administrative records to assess more 
completely the transition process and the extent to which Transition WORKS and the other five 
random assignment YTD projects improved transition outcomes. 

The Transition WORKS Project 

The Erie 1 Board of Cooperative Educational Services (BOCES), one of 37 regional public 
education service organizations that serve school districts throughout New York State, administered 
Transition WORKS. The project sought to maximize economic self-sufficiency and independence 
for youth with severe disabilities by improving their educational and employment outcomes. 
Transition WORKS was designed to fill identified gaps in existing transition services in the county. 
It provided training on self-determination and self-advocacy for youth and their families, transition 
planning, work-based experiences and other employment services, education support services, 
instruction on the organization of benefits-related paperwork, social and health services, and 
counseling on SSA benefits and waivers. 

Transition WORKS was staffed by nine employees of Erie 1 BOCES and additional individuals 
employed by three formal partner organizations. The assistant director for school support services at 
Erie 1 BOCES served as the project director. Her involvement diminished over time, but she 
continued to provide vision to the project. A full-time assistant project director was responsible for 
the day-to-day management of Transition WORKS, including coordination of services provided by 
the project partners and administration of the project’s management information system. Two full- 
time transition coordinators and five full-time job developers from Erie 1 BOCES provided the bulk 
of project services to youth, forming a closely knit team. Three full-time and three part-time staff of 
Neighborhood Legal Services delivered benefits counseling services. Smaller numbers of staff at the 
Community Employment Office and the Parent Network of Western New York provided 
participants with employment preparation services and their parents or guardians with training on 
organizing documents pertaining to SSA disability and other benefits. 
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Transition WORKS targeted a subset of the YTD-eligible population: youth ages 16 to 25 who 
were current or recent recipients of SSA disability benefits. Mathematica randomly selected Erie 
County youth satisfying these criteria from the SSA disability rolls and recmited them into the study 
starting in January 2007 and ending in March 2008. After the youth completed the baseline interview 
and provided written consent, we admitted them into the evaluation’s research sample. Mathematica 
then randomly assigned members of this sample to the evaluation’s treatment or control group at a 
six-to-five ratio, resulting in a treatment group of 459 youth and a control group of 384. 

At the time of random assignment, the youth in the research sample were about 20 years old, 
on average. A total of 62 percent of the study participants were male, 55 percent were white, and 
mental illness and cognitive or developmental disabilities were the primary disabling conditions for 
62 percent of the research sample. About half of the youth were in school at the time of random 
assignment, and about one in three reported having worked for pay during the previous year. 

Transition WORKS staff at Erie 1 BOCES were responsible for enrolling treatment group 
members in project services. Through an intensive effort from February 2007 through May 2008, 
they enrolled 380 youth, or 83 percent of the treatment group members. Youth were eligible to 
receive services for 18 months." Services were terminated in fall 2009, and the project formally 
ended in December 2009. 

Implementation Findings for Transition WORKS 

The original design for Transition WORKS specified a school/classroom-based intervention 
centered on a self-determination curriculum that encompassed career exploration activities but 
stopped short of delivering employment services to participating youth. That design was pilot tested 
in 2004 through 2006. To position Transition WORKS for participation in the YTD national 
random assignment evaluation, Erie 1 BOCES replaced the group-focused and classroom-based 
structure with a more individualized approach to transition services, broadened the project’s target 
population to include out-of-school youth, and expanded the program model to include the direct 
provision of employment services. The project retained its emphasis on youth empowerment 
through a self-determination workshop series based on the original classroom curriculum. Although 
the redesigned project was employment focused, the program model included an education services 
component in recognition of the need for some youth to participate in postsecondary or vocational 
education to prepare for their desired careers. 

Based on a well-developed program model that encompassed all major components of the 
YTD conceptual framework, Erie 1 BOCES and its partners delivered a structured sequence of 
services to the youth participating in Transition WORKS. The services began with an assessment of 
each participant’s level of self-determination, followed by two self-determination workshops. While 
youth were participating in those workshops, their parents or guardians were receiving instruction 
on organizing benefits-related paperwork. Upon completing the workshops, the participating youth 
engaged in benefits planning and general transition planning. At that point, they were considered 
ready to participate in education and employment-related services. As noted previously, there was 
considerable specialization among the partner organizations in the delivery of these services. 



2 Youth who enroll in YTD project services are eligible for the SSA waivers for four years following random 
assignment, or until age 22, whichever comes later. All wavier eligibility is scheduled to cease in September 2013. 
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Through our analysis of data from the Transition WORKS management information system, 
we found that nearly all (98 percent) of the youth who had agreed to participate in Transition 
WORKS received some project services. Two-thirds or more of them received services in each of 
the major program components noted above, with the exception of education services, which were 
not heavily emphasized by the project and were received by only 17 percent of participants. Among 
the youth who received any Transition WORKS services, the median number of service contacts 
was 10 and the median total duration of those contacts was eight hours. Half of the youth who 
received employment-related services had three or fewer service contacts with Transition WORKS 
staff for that purpose, lasting less than an hour in total. Several of the other services provided by 
Transition WORKS, such as self-determination training and benefits planning services, were 
somewhat more intense. Nearly 40 percent of the Transition WORKS participants could not recall 
having received project services at the time of the evaluation’s 12-month follow-up survey, but most 
who could recall receiving services were satisfied with them. 

Several aspects of the design and implementation of Transition WORKS may have limited the 
intensity of youth participation in services. First, the staff of the four partner organizations were not 
co-located; rather they were geographically dispersed by function, which may have been a barrier to 
participation in the full range of project services for some participants. Second, the structured 
sequence of project services, beginning with the workshops on self determination, meant that 
employment services typically were initiated about four months after youth enrolled in the project, 
which may have made it more challenging for staff to engage youth strongly in those services. 
Finally, project staff struggled to find time to serve enrollees adequately due to (1) the competing 
demand on their time to enroll treatment group members in the project and (2) large caseloads 
resulting from staff turnover and protracted staff vacancies. 

First- Year Impact Findings for Transition WORKS 

We estimated the impacts of Transition WORKS on outcomes in five domains: (1) 
employment-promoting services, (2) paid employment, (3) education, (4) youth income, and (5) 
attitudes and expectations. Within each domain, we analyzed one primary outcome and a number of 
secondary outcomes. The results for the primary outcomes are the basis for our principal 
conclusions regarding the project’s impacts in the year following random assignment. 

Impacts on the Use of Services 

Consistent with the YTD program model, Transition WORKS increased the use of employment- 
promoting services by youth with disabilities. Two-thirds of treatment group youth reported having used 
any employment-promoting service in the year following random assignment (Table 1). We 
estimated that, in the absence of Transition WORKS, slightly more than half of these youth would 
have used any such service. The impact of Transition WORKS was a 14 percentage point increase in 
the use of employment-promoting services. This overall impact was a product of impacts on the use 
of a number of specific types of employment services. The largest of these were on support for 
resume writing and job search activities and on benefits counseling. 

Transition WORKS also increased youth participation in non-employment services, such as life 
skills training and discussions of general interests, by eight percentage points (Table 1). Considering 
all types of services, 86 percent of treatment group members reported having used any employment 
or non-employment service. In the absence of Transition WORKS, we estimated that 77 percent of 
them would have used any service. Transition WORKS thus increased the share of youth using any 
service by nine percentage points. 



xviii 



Interim Report on Transition WORKS 



Executive Summary 



Table 1. Estimated Impacts of Transition WORKS on the Use of Services (percentages) 



Treatment Group 





Observed 

Mean 


Est. Mean w/o 
Transition 
WORKS 


Impact 




P- Value 


Domain: Employment- 


Promoting Services 








Primary outcome: used any employment- 
promoting service 


66.3 


52.6 


13.7 


*** 


0.00 


Used employment- promoting services: 












Career counseling 


37.7 


29.6 


8.1 


** 


0.02 


Support for resume writing and job search 


38.6 


23.3 


15.3 


*** 


0.00 


Job shadowing, apprenticeship/ internship 


15.6 


11.0 


4.7 


* 


0.07 


Other employment- focused services (basic skills 
training, computer classes, problem solving, 
and social skills training) 


5.0 


4.8 


0.3 




0.87 


Counseling on SSA benefits and work incentives 


33.8 


20.3 


13.5 


*** 


0.00 


Additional Service 


Use Outcomes 








Used any non- employment service 


81.5 


73.1 


8.4 


*** 


0.01 


Used any service (employment or non- employment) 


86.4 


77.1 


9.3 


*** 


0.00 



Source: YTD 12- month follow- up survey. 



Notes: The sample includes all youth who completed the study's 12- month follow-up survey. The table reports 

observed means or percentages for the treatment group, estimates of what the treatment group means or 
percentages would have been in the absence of Transition WORKS, and regression- adjusted impact estimates. 
We measured explanatory variables in the regression model prior to random assignment, using data from the 
study's baseline survey and SSA administrative records. We calculated all statistics using sample weights to 
account for interview non- response. The analysis sample includes 416 treatment group youth and 330 control 
group youth. Survey item non- response may have resulted in smaller sample sizes for specific outcomes. See 
Appendix Table A. 5 for the sample sizes for all outcomes. 

*/**/*** Imp act estimate is significantly different from zero at the .10/ .05/. 01 level using a two- tailed t- test. 



The previously mentioned positive impact of Transition WORKS on the use of benefits 
counseling services appears to have been reflected in greater knowledge of SSA work incentives 
among treatment group members. We estimated that Transition WORKS significantly increased 
awareness of each of five specific work incentives by between 7 and 18 percentage points (Table 
IV.3). However, this enhanced awareness of work incentives was not accompanied by greater 
understanding of the broader concept that benefits do not end as soon as a beneficiary begins 
working for pay. 

Impacts on Paid Employment and Other Key Outcomes 

Although Transition WORKS led to increased participation in employment-promoting services 
and in services more broadly defined, we did not find any significant impacts on the primary 
outcomes in the domains of paid employment, education, youth income, and attitudes and 
expectations (Table 2). 
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Table 2. Estimated Impacts of Transition WORKS on Employment and Other Key Outcomes in the 
Year Following Random Assignment (percentages, unless otherwise noted) 





Treatment Group 








Observed 

Mean 


Est. Mean w/o 
Transition 
WORKS 


Impact 


P- Value 


Domain: Paid Employment 


Primary outcome: ever employed in a paid job 


43.6 


40.7 


2.9 


0.39 


Total earnings 3 ' b 


$1,842 


$1,806 


$35 


0.89 


Domain: Education 


Primary outcome: ever enrolled in school or had 
completed high school by the end of the year 


82.0 


85.0 


-3.0 


0.22 


Domain: Youth Income 


Primary outcome: total annual income (earnings and SSA 
benefits) 3 ' b 


$9,013 


$8,830 


$183 


0.55 


Number of months of benefit receipt 


11.3 


11.2 


0.1 


0.30 


Total SSA benefit amount 


$7,142 


$6,993 


$149 


0.40 


Domain: Attitudes and Expectations 


Primary outcome: youth agrees that personal goals 

include working and earning enough to stop receiving 
Social Security benefits 


67.3 


69.7 


-2.4 


0.53 



Sources: YTD 12- month follow- up survey and SSA administrative records. 



Notes: The sample includes all youth who completed the study’s 12- month follow-up survey. The table reports 

observed means or percentages for the treatment group, estimates of what the treatment group means or 
percentages would have been in the absence of Transition WORKS, and regression- adjusted impact estimates. 
We measured explanatory variables in the regression model prior to random assignment using data from the 
study's baseline survey and SSA administrative records. We calculated all statistics using sample weights to 
account for interview non- response. The analysis sample includes 416 treatment group youth and 330 control 
group youth. Survey item non- response may have resulted in smaller sample sizes for specific outcomes. See 
Appendix Table A. 5 for the sample sizes for all outcomes. 

a For these outcomes, item non-response occurred conditionally, depending on the values of other measures in the 
follow-up survey. The rate of missing data is 10 percent for both earnings and income. We used a "multiple 
imputations" procedure to assign values when they were missing. See Appendix A, Section E, for more information on 
this procedure. 

The average includes youth who were not employed during the year following random assignment. 



Our primary outcome in the domain of paid employment was whether a youth was ever employed in 
a paid job during the year following random assignment. We found that 44 percent of treatment 
group youth worked for pay sometime during the year but we estimated that this outcome would 
have been essentially the same in the absence of Transition WORKS. We also estimated the impact 
on total earnings during the year but found none. In summary, although Transition WORKS 
increased the receipt of employment-promoting services, that did not translate into impacts on paid 
employment within the first year of program experience. 

Education services were not central to the Transition WORKS program model. Thus, we were 
not surprised to find that the project had no impact on our primary outcome in the domain of 
education , which was whether a youth was ever enrolled in school during the year following random 
assignment or had successfully completed high school by the time of the 12-month survey. 
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Interim Report on Transition WORKS 



Executive Summary 



In the domain of youth income , we found that Transition WORKS had no impact on the primary 
outcome: total youth income from earnings and benefits. Furthermore, although the intervention 
did improve knowledge of SSA work incentives and requirements, that did not translate into 
treatment group youth receiving more benefits than they would have otherwise. We found no 
impact on the number of months of benefit receipt during the year following random assignment or 
on the total amount of benefits received during that year. 

Finally, we found that Transition WORKS had no impact on the primary outcome in the domain 
of attitudes and expectations. Table 2 shows that about two-thirds of treatment group youth agreed that 
their personal goals included working and earning enough to stop receiving disability benefits. 
However, we estimated that this proportion would have been essentially the same in the absence of 
Transition WORKS. 

Conclusion 

The YTD evaluation seeks to inform SSA about transitions by youth with disabilities to 
employment and adult life, and the ways in which they could be facilitated. More specifically, this 
evaluation is testing whether a well-defined set of services and enhanced SSA work incentives can 
increase employment and self-sufficiency among youth with disabilities. In the case of Transition 
WORKS, this approach did result in greater use of services to promote employment; however, the 
project had no impacts on youth employment, education, income, and expectations during the one- 
year follow-up period for this report. The structured sequence of project services, geographic 
dispersion of service providers by function, and large caseloads for key project staff may have made 
it challenging for project staff to provide participating youth with services that were sufficiently 
intense to result in improvements in these measures during the follow-up period. 

It is important to recognize that this report has presented interim impact estimates based on 
data pertaining to the first year in the evaluation’s multiyear follow-up period; many of the enrolled 
youth were still receiving project services when they completed the 12-month follow-up survey. 
Interim evaluation findings from the other five random assignment YTD projects will enable us to 
extend the initial assessments presented in this report. Interim reports on two of those projects, 
along with this report on Transition WORKS, will be completed in 2011, while the interim reports 
on the remaining three projects will be completed in 2012. As was planned, the projects vary in the 
mix and intensity of services while broadly adhering to the YTD program model. Therefore, we 
expect that the full set of six interim evaluation reports will provide SSA with a better understanding 
of the challenges that youth with disabilities face in making transitions and the specific types of 
interventions that might assist more of them to succeed. Furthermore, the YTD evaluation’s 
comprehensive final report will present impact estimates based on 36 months of follow-up data 
from all six of the random assignment projects. Our analyses of those data may reveal longer-term 
impacts of Transition WORKS in addition to the short-term impacts reported here. 
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